7 research outputs found
Accelerating Training of Deep Neural Networks via Sparse Edge Processing
We propose a reconfigurable hardware architecture for deep neural networks
(DNNs) capable of online training and inference, which uses algorithmically
pre-determined, structured sparsity to significantly lower memory and
computational requirements. This novel architecture introduces the notion of
edge-processing to provide flexibility and combines junction pipelining and
operational parallelization to speed up training. The overall effect is to
reduce network complexity by factors up to 30x and training time by up to 35x
relative to GPUs, while maintaining high fidelity of inference results. This
has the potential to enable extensive parameter searches and development of the
largely unexplored theoretical foundation of DNNs. The architecture
automatically adapts itself to different network sizes given available hardware
resources. As proof of concept, we show results obtained for different bit
widths.Comment: Presented at the 26th International Conference on Artificial Neural
Networks (ICANN) 2017 in Alghero, Ital
Morse Code Datasets for Machine Learning
We present an algorithm to generate synthetic datasets of tunable difficulty
on classification of Morse code symbols for supervised machine learning
problems, in particular, neural networks. The datasets are spatially
one-dimensional and have a small number of input features, leading to high
density of input information content. This makes them particularly challenging
when implementing network complexity reduction methods. We explore how network
performance is affected by deliberately adding various forms of noise and
expanding the feature set and dataset size. Finally, we establish several
metrics to indicate the difficulty of a dataset, and evaluate their merits. The
algorithm and datasets are open-source.Comment: Presented at the 9th International Conference on Computing,
Communication and Networking Technologies (ICCCNT
DLKoopman: A deep learning software package for Koopman theory
We present DLKoopman -- a software package for Koopman theory that uses deep
learning to learn an encoding of a nonlinear dynamical system into a linear
space, while simultaneously learning the linear dynamics. While several
previous efforts have either restricted the ability to learn encodings, or been
bespoke efforts designed for specific systems, DLKoopman is a generalized tool
that can be applied to data-driven learning and optimization of any dynamical
system. It can either be trained on data from individual states (snapshots) of
a system and used to predict its unknown states, or trained on data from
trajectories of a system and used to predict unknown trajectories for new
initial states. DLKoopman is available on the Python Package Index (PyPI) as
'dlkoopman', and includes extensive documentation and tutorials. Additional
contributions of the package include a novel metric called Average Normalized
Absolute Error for evaluating performance, and a ready-to-use hyperparameter
search module for improving performance